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ABSTRACT 

A computer program was written that carries out reliabi- 
lity assessments according to a method proposed by the TRW 
Corporation. This method combines point estimates for 
reliability from different sources into an overall point 
estimate. The program was used to calculate the overall 
point estimate for cases covering a range of sample sizes, 
and underlying probabilities of success in order to make a 
judgement on the usefulness of the method. 
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I. 



INTRODUCTION 



In deriving a point estimate for the reliability of a 
system or component, there is no universally accepted method 
for combining data from more than one test environment. For 
example, if a system on a test stand performs its intended 
function four times out of five trials, and in another 
environment, say actual use, it performs its function in two 
out of three trials, the point estimates for system reliabi- 
lity in the two environments would be 4/5 and 2/3 respectively 
It is not clear how these two estimates from different environ 
ments should be combined into a single point estimate of 
realiabiltiy . 

The Quality Evaluation Laboratory, U.S. Naval Ammunition 
Depot, Oahu, Hawaii, published a procedure to accomplish this 
integration of trial data by comparing all data sets to a 
base data set and generating weighting factors for each data 
set. TRW Corporation wrote a computer routine to carry out 
the required computations for this method and proposed the 
method be adopted use in certain development programs. 

The TRW computer program was not available for this 
study and so two programs were written as part of this project 
Each program has certain limitations in use, but together they 
cover a complete range of method parameters. 
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II. THE METHOD 



Consider a series of n trials of a system of which m are 
judged successes. Then the probability that R is the under- 
lying reliability of the system is given by the binomial 
distribution . 



P(R) = n C m (0<R^1). 

This relationship generates a probability density function 
such as is shown in figure 1. This is the likelihood function 
of R given m successes in n trials. 




0 R I 



FIGURE 1. Likelihood function 
of R given m successes in n 
trials . 



To evaluate percentiles of the binomial distribution, the 
incomplete beta function'*" can be used through the following 
identity : 

n 

I (a,n-a+l) = £ C R x (l-R) n x , n and k positive integers. 

x\ XI X 



The incomplete beta function is defined 

.x 



I x (a ' b) B (a,b) f 0 



t a 1 (l-t) b 1 dt, (0<t<l) 



where 



B (a , b) 



r (a) r (b) 
r (a+b) 



The incomplete beta family is extensively tabled and efficient 
comouter routines have been written to compute I (a,b) . 
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If the probability density function for a second set of 
test data is superimposed on the first density function, a 



representation such as figure 2 results. 



P (R) 




FIGURE: 2. Two likelihood 
functions showing overlap. 



0 



R 



The TRW method uses the amount of overlap as a weighting 
factor that measures the 'extent of agreement between the two 
sets of data. Calculating the weighting factor can be done 
by first solving for points of intersection of the two curves 
These are all points x which are solutions to the equation 



where 

n = number of trials in data set 1 
m = number of successes in data set 1 
p = number of trials in data set 2 
s = number of successes in data set 2 
This expression simplifies to 



This is a polynomial of degree n-p. 

With the points of intersection determined, the TRW 
method determines the weighting factor by summing the area 
under that curve which is the lower between each intersection 
See figure 2. 




x 



m-s 



(1-x) 



n-m-p+s _ pi (n-m) 1 ml 
~ (p-s) 1 s 1 nl 
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The two computer programs included in this report perform 
these calculations. Limitations on the programs are discussed 
in the descriptions that introduce each program. 

To calculate a point estimate of reliability using the 
TRW method, each set of data, i = l,2,...,n, is compared with 
a set of base data and a weighting factor, k^, is determined 
as discussed above. These weighting factors are then applied 
to the point estimates to determine the overall estimate as 
follows : 



R 



l.O(P') + k 1 (P£) + k 2 (P£) + 




1 . 0 + k. + k 0 ... + k 
12 n 



where 



R = overall point estimate of reliability 

th 

P£ = point estimate from i data set 

P^ = point estimate from base data set 

th 

k. = weighting factor from i data set 



i 



i =1,2, . . . , n 
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III. ANALYSIS OF TRW METHOD 



The magnitude of the weighting factor or overlap used in 
this method is determined by the relationship between the two 
incomplete beta functions used to represent the data sets. 
Specifically, the overlap is a function of (1) the location 
of the mean of each curve relative to the mean of' the other 
end and (2) the spread or variance of each curve. ^ A parti- 
cular realization of the beta function tends to become spike 
shaped as the number of trials is increased. This effect is 
shown in table 1 which lists by way of example seven data 
sets, all having a mean of .8000. 

Number of trials Beta curve Variance 



9 


B (x; 5, 4) 


.0246 


18 


B (x; 10, 8) 


.0129 


25 


B (x; 20 , 5) 


.0062 


40 


B (x; 32,8) 


. 0039 


50 


B (x; 40 , 10) 


.0031 


60 


B (x ; 48 , 12 ) 


. 0026 


100 


B (x; 80 , 20) 


.0016 



Table 1. As the number of trials is increased, the 
variance of the beta distribution representation decreases. 
The means of all data sets is .8000. 

It was hypothesized that the weighting factor would be rela- 
tively small (and consequently the TRW method would have only 



The mean and variance of the beta distribution representing 

a data set with s successes and f failures are 

mean = f variance = 0 S ' x -- — . 

(s+f) (s+f) 2 X (s+f+1) 
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a minor effect on reliability calculations) when (1) the 
ratios of successes to total trials from each data set were 
widely displaced from each other, or (2) when the number of 
trials in either data set was not small and the ratio of 
successes to number of trials were more than a little displaced 
from each other. 

To investigate this idea, pairs of test data sets and 
base data sets were compared and a weighting factor and point 
estimate of reliability was determined using the TRW method 
and the computer routines written for this project. A range 
of probabilities of success on a single trial and total 
number of trials was considered for both base and test data 
sets. Thirty six pairs were considered. 

Table 2 is a tabulation of the data from the investigation. 
The number of trials in the base set, n^, and in the trial 
data set, n fc , are shown for each case as are the underlying 
probabilities of success on a single trial, P^ for the base 
data and for the trial data. Each of the ten replications 
using these parameters resulted in a number of successes, s^ 
and s t , and point estimates of reliability, P£ and P£. These 
last were calculated as 




The number of successes and the number of failures in each 
replication were inputs into the computer program which 
generated the TRW weighting factor, K. The TRW point estimate 
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Base Data Sets Test Data Set 
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Table 2 (continued) 



of reliability, R, was then calculated as 



1,0 x P/ + K x P' 
b t 



The magnitude of the adjustment to the base data point esti- 
mate is the absolute value of the difference R-P£. The 
values generated in the ten replications were averaged and 
these average values, i^, P£/ s" t , P£, K, R, and | iR-P ^ | r are 
the entries in table 2. 

By way of example, consider case 1. In this case, the 
base data set consisted of ten trials (n^) , each with a 
probability of success of .99 (P£) . The test data set con- 
sisted of twenty trials (n t ) , each with a probability of success 
of .80 (P t ) . The results of each of the ten replications are 
shown below in table 3. (Data from all replications is con- 
tained in the computer output section of this report.) 



Replication 


s b 


P b 




s t 


P i 


TRW K 


R 


l R - p bi 


1 


9 


.90 




16 


.80 


.5024 


.8666 


.0334 


2 


9 


.90 




16 


.80 


.5024 


. 8666 


.0334 


3 


9 


.90 




17 


.85 


.6641 


.8800 


.0200 


4 


9 


.90 




17 


.85 


.6641 


. 8800 


.0200 


5 


9 


.90 




17 


.85 


.6641 


.8800 


.0200 


6 


9 


.90 




17 


.85 


.6641 


.8800 


.0200 


7 


9 


.90 




18 


.90 


.8673 


.9000 


.0000 


8 


9 


.90 




18 


.90 


.8673 


.9000 


.0000 


9 


9 


.90 




13 


.65 


.2019 


.8580 


.0420 


10 


9 


.90 




15 


.75 


.3731 


.8592 


.0408 


Averages 


9.0 


. 90 




16.4 


.82 


.5971 


.8771 


.0229 


Table 3 . 


Example showing 


the generation 


of data 


for 


Table 2, 


Case 1 


; Base 


Data 


Set: n, 
b 


= 10, 


= .99, Test 


Data Set 


: n t = 


20, P 


t 


= .80. 








The last row 


in table 3 


shows 


the average figures that are 


the table 2 


entries 


for 


case 1 


. Table 


2 alsc 


> lists 


the standard 



deviation of the distribution of R for each case. 
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The data of table 2 shows that the TRW method consistently 
makes a correction to the point estimate of reliability that 
is in the proper direction. The largest correction to the 
point estimate in any of the 360 computations was .042. 

This occurred when the base data was nine successes out of 
ten trials and the test data was thirteen successes out of 
twenty trials. As hypothesized, as the number of trials in 
either data set increased, or as the point estimates from 
the data sets diverged, the correction to the point estimate 
decreased. Whenever the point estimates coincided, there was 
no correction. 
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IV. CONCLUSIONS 



The correction that the TRW method applies to a base data 
point estimate decreases as the number of trials in either 
data set increases or as the point estimates from the data 
sets diverge. The method probably would adjust the point 
estimate less than would an observer making a subjective 
judgement based on the data. 
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PROGRAM ONE 



PURPOSE 

SOLVES FOR WEIGHTING FACTOR USED IN TRW METHOD OF 
CALCULATING OVERALL RELIABILITY". OUTPUT IS THF 
POINTS WHERE TWO B^TA DENSITIES INTERSECT, THE AREA 
UNDER EACH CJPVE FROM ORIGIN TO THOSE POINTS (CALLED 
P(X) ), THE HEIGHT OR DENSITY AT THOSE POINTS, AND 
THE WEIGHTING OR K FACTOR. 

METHOD 

RTPLBS IS USED TO SOLVE THE POLYNOMIAL 
X**(A-C )“(1-X)**(B-D) = Y 
WHERE 



Y= ( { C+D-l ) " ( A- 1 ) ■' { B- 1 ) )/<(A + B-l) T(C-l) *(D-1> ) 

IF A-C AND 3-D ARE BOTH NEGATIVE THEN THE POLYNOMIAL 

X**(C-A)*l 1-X)**(D-B) = 1/Y 

SHOULD BE SUBSTITUTED. IF ONLY ONE OF THESE TERMS 
IS NEGATIVE USE PROGRAM TWO. 

SOLUTIONS BETWEEN 0 AND 1 ARE INPUTS FOR BDT R WHICH, 
WITH SUBSEQUENT PROGRAM LOGIC, SOLVES FOR WEIGHTING 
FACTOR. 

PARAMETERS 

COEF-A VECTOR OF COEFICIENTS OF POLYNOMIAL. MUST 
3c DECLARED AS REALMS OF DIMENSION N+I . 

A -NUMBER OF SUCCESSES IN BASE DATA. 

B -NUMBER OF PA I LURES IN BASE DATA. 

C -NUMBER OF SUCCESSES IN TEST DATA. 

D -NUMBER OF FAILURES IN TEST DATA. 

N -DEGREE DF POLYNOMIAL USED TO DETERMINE POINTS 
OF INTERSECTION. 

Y -CONSTANT TERM IN POLYNOMIAL. 

THE TERMS DF THE POLYNOMIAL MUST BE WORKED OUT 
MANUALLY AND MUST BE SUPPLIED THROUGH VECTOR 
COEF. Y MUST BE SUPPLIED AS FIRST STEP 0~ THE 
PROGRAM. MOST OF THF TERMS OF Y CANCELL. AN ARITH- 
METIC EXPRESSION CAN BE SUPPLIED TO COMPLETE THE 
CALCULATIONS. AFTER Y IS ENTERED THE PROGRAM SETS 
C0EF(N+1) = Y. 

INSERT DIMENSION OF VECTOR COEF AS N+l IN TYPE STATE 
MENT AT LINE 0UQ7 OF PROGRAM. 

INSERT PARAMETERS A,B,C,D,M, AND VECTOR COEF IN 
DATA STATEMENT AT LINO 0008 OF PROGRAM. 

I NS EP T PARAMETER Y AT LINE 0009 OF PROGRAM. 

LIMITS ON USE OF PROGRAM 
N MUST BE LESS THAN A9 

EXPONENTS OR POLYNOMIAL MUST NOT BE NEGATIVE 



IF EITHER OF THESE LIMITATIONS ARE NOT MET USE 
PROGRAM TWO AND C P-67 /CMS TIME SHARING SYSTEM. 



SUBROUTINES REQUIRED TO 3E SUPPLIED BY USER 
RTPLSB 
BDT D 
DLGAM 
NDTP 
CDTR 
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* 








non o o 



CALL EPRSET (207,300,-1,1) 

1000 FORMAT ( ' 1 1 , ' COMPUTED CONSTANT IS • ,D13.6) 

1100 FQRMA T ( ’ O' » 5X, 'ROOT IS ',D23.6) 

1200 FORMAT ( 'O' ,5X,3D23.6) 

1700 FORMAT ( 'O' , 5X, ' FOR X= *,E13.6,' A BETA ( ' , F5 . 1 , ' ,', 

2F5.1, ' ) HAS P(X) ' , F 1 0 . 7 , ' AND DENSITY ',F10.7 

3 / 1 X , ' THE ERROR CODE IS ',14//) 

1800 FORMAT ('O', ' K FACTOR IS * , F10.7) 

0007 RE AL-8 COEF( ) , U ( 49 ) , V ( 49 ) , CONV ( 49 ) , Y, ROOT ( 5 ) , 

2DEM1( 2) ,DEN2< 2 ) , PR031 (2) ,PR062 (2) 

0008 DATA COEF/ / 

2U, V,C0NV/49*0. 0,49*0. 0, 49*0.0/ , ROOT/ 5*0. 0/ , 

3PR0B 1/2*0. 0/,PR082/0.0/ , DENI/ 2 *0.0/ , DEN2/2*0 .0/, N/ / 

4 , A, B , C , 0/ / 

0009 Y = 

Y = - Y 

WRITE (6,1000) Y 
M = N + 1 
I = 1 
COEF ( M ) =Y 

CALL RTPLSB TO GET ROOTS AND DETERMINE REAL ROOTS 

CALL RTPLSB ( N , COE F , U , V , CONV , I ER ) 

DO 25 J = 1 , N 
IF ( V( J ) ) 25,21,25 
21 IF ( U { J ) ) 25,25,26 

26 IF (U(J) - 1.0) 27,27,25 

27 ROOT ( I ) = U ( J ) 

I=Ii-l 

25 CONTINUE 

IF (ROOT (1 )-ROOT (2) ) 30,30,31 
31 TF MP = ROOT ( 1 ) 

ROOT ( 1 )=R00T(2 ) 

ROOT (2 ) =T E MP 
30 I I = I - 1 
DO 50 :< =1,2 

CALL 3 PTR ( ROOT ( K ) , A , B , PRCB1 ( K ) , DENI ( K ) , I ER ) 

WRITE ( 6 , 17C0 ) ROOT ( K ) , A , 8 , PROB 1 ( K ) , DENI ( K ) ,IER 
CALL BPTR( ROOT ( K ) , C , 0 , PR082 ( K ) , DEN 2( K ) , I ER ) 

WRITE (6,1700) ROOT (K) ,C ,0 ,PRP62 <K) ,DEN2 (K ) , IER 
50 CONTINUE 

IF { PR0B1 (1 ) -PR0B2 ( 1 ) ) 52,54,53 

52 Z = PR03 1 ( 1) + PR0B2(2) - D R0B2( 1 ) +1 . O-oROBl ( 2 ) 

GO TO 60 

54 I F ( PR0B1 ( 2 ) - PR0B2 ( 2 ) ) 53,53,52 

53 Z=PR032( l)+ p R331(2)-PR031( 1)+1.0-PRQB2(2) 

60 WRITE (6, 1800) Z 

END 
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PROGRAM TWO CP-67/CMS NAME FOR PROGRAM IS EVAL2 
PURPOSE 

WHEN PROGRAM ONE IS NOT USED TO SOLVE FOR WEIGHTING 
FACTOR, THIS SIMPLE LOOK UP TYPE PROGRAM CAN BE USED 
WITH CP- 67/CM S TO CONVERGE TO POINTS OF INTER 
SECTION OF TWO BETA CURVES AND TO GET AREAS UNDER 
THE CURVES FROM THE ORIGIN TO THE POINTS OF 
INTERSECTION. CALCULATION O c THE WEIGHTING FACTOR 
CAN THEN BE DONE BY HAND AFTER OBSERVING WHICH CURVE 
IS LOWER TO THE LEFT OF THE INTERSECTION. 

METHOD 

THE PARAMETERS FOR TWO BETA DISTRIBUTIONS ARE READ 
IN AS INPUTS. ALSO INPUTS BY A READ STATEMENT 
SUPPLIED TO SUBROUTINE BDTR WHICH SUPPLIES AS OUTPUT 
THE DENSITIES OR HEIGHTS OF THE TWO CURVES AT THE 
INPUT POINTS. THE USER CAN OBSERVE CONVERGENCE 
ON INTERSECTIONS (POINTS OF EQUAL HEIGHT ON BOTH 
CURVES). HE CAN THEN READ IN NEW VALUES FOR INPUT 
POINTS UNTIL H F LOCATES THE INTERSECTIONS. THIS CAN 
BE DONE QUITE RAPIDLY WITH TIME SHARING. 

PARAMETERS 

A-NU V '8ER OF SUCCESSES IN BASE DATA. 

B-NUMBER OF FAILURES IN BASE DATA. 

C-nuvbfr of SUCCESSES IN TEST DATA. 

D-NUM8ER OF FAILURES IN TEST DATA. 

S-ORDINATE AT WHICH BETA CURVES ARE EVALUATED. 

T-CRDINATE AT WHICH BETA CURVES ARE EVALUATED. 

U-DRDINATE AT WHICH BETA CURVES ARE EVALUATED. 

V-ORD IHAT E AT WHICH BETA CURVES ARE EVALUATED. 

W-ORPINATE AT WHICH BETA CURVES ARE EVALUATED. 

X-ORDINATE AT WHICH 3ETA CURVES ARE EVALUATED. 

Y-ORDIMATE AT WHICH BETA CURVES ARE EVALUATED. 

Z-ORDI NATE AT WHICH BETA CURVES ARE EVALUATED. 

SUBROUTINES REQUIRED TO BE SUPPLIED BY USER 
BDTR 
DL GAM 
NDTR 
CDTR 



FORTRAN 



OFFLINE READ EVAL2 
800 FORMAT ( 

900 FORMAT ( 4F4. 0 ) 

1700 FORMAT ( IX, 'FOR X= 
2F5.1,' ) HAS P(X) • 

3,14) 

RE AD ( 5 , 9 DO ) A,8,C,F 

RE AD ( 5 , 3 0 0 ) S , T , U , V , W ,X,Y,Z 

CALL 80TP( S,A,B ,P,D, TER) 



' , E 1 3 . 6 , ' A 
, F 1 0 . 7 , ' 



BETA ( • ,F5. 1, ' 

AND DENSITY ' , F10.7 



, ' , 



WRITE (6 , 1700 ) S, A, E,P, D, IER 
CALL BDTR ( S,C,” ,P,D,IER) 
WRITE (6,1700) S,C,F,P,D,IER 
CALL BOT R ( T , A , 3,P, D, IFR) 

WRI TE ( 6 , 1700) T,A,B ,P, D, IER 
CALL BDTR ( T , C , p ,P, n , I C R) 

WRI TE ( 6 , 1 700 ) T, C, F t p, D, IER 
CALL B D T R ( U , A , B , P , D , IER) 
WRITE (6 , 1700 ) U , A, 3 , P , D, IER 
CALL 9 D T P ( U , C , F ,P,D, IER) 
WRITFU, , 1700) U ,C , F ,P, D, I ER 
CALL 8DTR(V, A, 3,P, D, IER) 
WRITE ( 6 , 17 00) 7,A,B,P,D,IER 
CALL BDTR ( V , C , c , P , D, IER) 
WRITE (6, 17 3D) V , C , F , P , D , I ER 
CALL BDTR(W,A, 3 ,P,D, IER) 
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WRITE(fc» 1700 ) ^,4,P,P,0, 
CALL BDTR(W,C, F ,P,D, IER) 
WRITE ( 6, 1700) W,C,F,P,D, 
CALL BDTR{X,A,3,P,P, IER) 
WRI TE ( 6 , 17 3 0 ) X,A,B,P,D, 
CALL BDTR{X,C, <= ,P,D, IER) 
WRITE (6 ,1700) X , C , F , P , D , 
CALL BDTR( Y » A , B » P » D , I ER ) 
WRITE16, 1700 ) Y , A , B , P , P» 
CALL BDTR(Y,C»F,P,D,IER) 
WRITE! 6, 1700) Y,C,F,P,D, 
CALL BDT R(Z,A»B»P,D, IER) 
WRITE ( 6 , 17 30) Z,A,B,P,D, 
CALL BDTR( Z ,C, F ,P, B, IER) 
WRITE (6 ,1700) Z , C , F , P , D, 
END 



IER 

IER 

IER 

IER 

IER 

IER 

IER 

IER 
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